Reliable Confidence Predictions Using Conformal Prediction

نویسندگان

  • Henrik Linusson
  • Ulf Johansson
  • Henrik Boström
  • Tuve Löfström
چکیده

Conformal classifiers output confidence prediction regions, i.e., multi-valued predictions that are guaranteed to contain the true output value of each test pattern with some predefined probability. In order to fully utilize the predictions provided by a conformal classifier, it is essential that those predictions are reliable, i.e., that a user is able to assess the quality of the predictions made. Although conformal classifiers are statistically valid by default, the error probability of the prediction regions output are dependent on their size in such a way that smaller, and thus potentially more interesting, predictions are more likely to be incorrect. This paper proposes, and evaluates, a method for producing refined error probability estimates of prediction regions, that takes their size into account. The end result is a binary conformal confidence predictor that is able to provide accurate error probability estimates for those prediction regions containing only a single class label.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

On Improving Breast Cancer Prediction Using GRNN-CP

The aim of this study is to predict breast cancer and to construct a supportive model that will stimulate a more reliable prediction as a factor that is fundamental for public health. In this study, we utilize general regression neural networks (GRNN) to replace the normal predictions with prediction periods to achieve a reasonable percentage of confidence. The mechanism employed here utilises ...

متن کامل

Intelligent Systems Reliable diagnosis of acute abdominal pain with conformal prediction

Medical decision support is an area in which a lot of machine learning research has been conducted and several diagnostic and prognostic systems have been developed. The majority of these systems only produce bare predictions, without any indication of how reliable each of these predictions is. An indication of this kind however, is highly desirable especially in the medical field. In this pape...

متن کامل

Evaluation of the Risk of Stroke with Confidence Predictions Based on Ultrasound Carotid Image Analysis

Conformal Predictors (CPs) are Machine Learning algorithms that can provide reliable confidence measures to their predictions. In this work, we make use of the Conformal Prediction framework for the assessment of stroke risk based on ultrasound images of atherosclerotic carotid plaques. For this application, images were recorded from 137 asymptomatic and 137 symptomatic plaques (symptoms are St...

متن کامل

Reliable Yield Prediction with Regression Neural Networks

This paper performs an extension to conventional regression neural networks (NNs) for replacing the point predictions they produce with prediction intervals that satisfy a required level of confidence. Our approach follows a novel machine learning framework, called Conformal Prediction (CP), for assigning reliable confidence measures to predictions without assuming anything more than that the d...

متن کامل

Applying Mondrian Cross-Conformal Prediction To Estimate Prediction Confidence on Large Imbalanced Bioactivity Data Sets.

Conformal prediction has been proposed as a more rigorous way to define prediction confidence compared to other application domain concepts that have earlier been used for QSAR modeling. One main advantage of such a method is that it provides a prediction region potentially with multiple predicted labels, which contrasts to the single valued (regression) or single label (classification) output ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2016